Arabic Morphosyntactic Raw Text Part of Speech Tagging System Phd Dissertation Chapter 4

We present a comprehensive Arabic tagging system: from the raw text to tagging disambiguation. For each processing step in the tagging system, we analyze the existing solutions (if any) and use one of them or propose, implement and evaluate a new one. This work began with designing a new Arabic tagset suitable for Classical Arabic (CA) and Modern Standard Arabic (MSA). In addition to the classical constructions in tag systems, we introduce interleaving of tags. Interleaving is a relation between tags which, in certain situations, can be attached to the same occurrence of a word, but each of them can also appear alone. Our tagset makes this relation explicit. Then we deal with the preparatory stages for tagging system. The first initial stage is tokenization and segmentation. We use rule-based and statistical methods for this task. The second stage is analyzing and extracting the lemma from the word. We have created our own analyzer compatible with our requirements. Its main part is a dictionary which provides features, POS and lemma for each word. The last part of our work is the tagging algorithm which produces one tag for each word. We use a hybrid method by combining rules-based and statistical methods. Three taggers, Hidden Markov Model (HMM), maximum match and Brill are combined by a new method, which we call master and slaves. Then handwritten rule-based tagger is also added to master-slaves. The rule based tagger eliminates incorrect tags, and the master chooses the best one among the remaining ones, assisted by the other slaves. Our complete system is ready to be used for annotation of Arabic corpora.